Stochastic Tree Ensembles for Regularized Nonlinear Regression

This article develops a novel stochastic tree ensemble method for nonlinear regression, referred to as accelerated Bayesian additive regression trees, or XBART. By combining regularization and search strategies from modeling with computationally efficient techniques recursive partitioning algorithms, XBART attains state-of-the-art performance at prediction function estimation. Simulation studies demonstrate that provides accurate point-wise estimates of the mean does so faster than popular alternatives, such BART, XGBoost, neural networks (using Keras) on variety test functions. Additionally, it is demonstrated using initialize standard BART MCMC algorithm considerably improves credible interval coverage reduces total run-time. Finally, two basic theoretical results are established: single version model asymptotically consistent Markov chain produced by has unique stationary distribution.